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Abstract. We extend the Dirichlet principle to non-reversible Markov pro- 
cesses on countable state spaces. We present two variational formulas for the 
solution of the Poisson equation or, equivalently, for the capacity between two 
disjoint sets. As an application we prove a some recurrence theorems. In par- 
ticular, we show the recurrence of two-dimensional cycle random walks under 
a second moment condition on the winding numbers. 



1. Introduction 

Since Kakutani work |10) , probability theory has not only proven to be a powerful 
tool inside potential theory, but potential theory has also given deep insight into the 
study of Markov processes. For example, the Dirichlet and the Thomson principles, 
which express escape probabilities as inhma and suprema, respectively, give efficient 
recurrence and transience criteria. One can use the Dirichlet principle to prove the 
recurrence of random walks in random conductances in dimension one and two, 
and the Thomson principle to prove transience in dimension larger than or equal 
to three [322 [[7]. 

Recently, potential theory and the Dirichlet principle played an important role 
in the proof of almost sure convergence of Dirichlet functions in transient Markov 
processes [1] [6], and in the proof of the recurrence of a simple random walk on 
the trace of transient Markov processes [5]. In a completely different context, the 
Dirichlet principle has been a basic tool in the investigation of metastability of 
reversible Markov processes (cf. [11 [3] and references therein). 

Most applications of potential theory to Markov processes, as the ones cited 
above, are however restricted to reversible processes due to the lack of variational 
formulas for the effective resistance between two sets in non-reversible processes. 
We fill this gap here, presenting a Dirichlet principle for general Markov processes 
in discrete state spaces. 

To illustrate the interest of the Dirichlet principle, we present some direct impli- 
cations of this result. In Lemma 1^751 we extend to non-reversible transient Markov 
processes a well known pointwise estimate of a function in terms of its Dirichlet 
form and the Green function. In the last section, we state some recurrence theorems 
for non-reversible processes. In particular, we show that the recurrence property of 
Durrett multidimensional generalization [8] of Sinai random walk relies in fact on 
the scale invariance properties of a stationary measure and not on the reversibility 
of the process. We also give a sufficient second moment condition for the recurrence 
of two-dimensional cyclic random walks considered before in [TTJ Q21 [7j [12] ■ 
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In a completely different direction, relying on the Dirichlet principle presented in 
this article we prove in [9] the metastable behavior of the condensate in supercritical 
asymmetric zero range processes, extending to the non- reversible case the results 
proved in [3]. 

2. Notation and Main results 

Consider an irreducible Markov process {Xt : t > 0} on a countable state space E 
with generator L. Denote by A(x), x G E, the holding rates, by p(x, y), x ^ y G E, 
the jump probabilities, and by r(x,y) = X(x)p(x,y) the jump rates. In particular, 
for every function / : E — > R with finite support, 

(Lf)(x) = 2/) [/(*/)-/(*)] , *^E. (2.1) 

y£E 

Note that Lf is well defined for a bounded function /. 

Assume that the Markov process {X t : t > 0} admits a stationary state \i. Let 
L 2 (/i) be the space of square summable functions / : E — > R endowed with the 
scalar product defined by 

(f,g)n = f{x)g{x) . 

Denote by the same symbol L the generator acting on a domain of L 2 (/i), and by 
D(f) the Dirichlet form or energy of a function / : E — > R : 

D (f) = \ E ^)K^i/)[/(i/)-/(^)] a 

x,y£E 

so that for / in the domain of the generator we have D(f) = (f, (— L)f)^. 

For each x G E, denote by P x the probability measure on the path space 
D(R + ,E) of right continuous trajectories with left limits induced by the Markov 
process X t starting from x. Expectation with respect to ¥ x is denoted by E^. 

Denote by {X? : t > 0} the stationary Markov process X t reversed in time. We 
shall refer to X t * as the adjoint or the time reversed process. It is well known that 
XI is a Markov process on E whose generator L* is the adjoint of L in L 2 (fi). 
The jump rates r*(x, y), x ^ y G E, of the adjoint process satisfy the balanced 
equations 

/j,(x)r(x,y) = n(y)r*(y,x) . (2.2) 

Denote by X*(x) — X(x), x G E, p*(x,y), x ^ y G E, the holding rates and the 
jump probabilities of the time reversed process X£ . 

As above, for each x G E, denote by P* the probability measure on the path 
space D(M. + ,E) induced by the Markov process X t * starting from x. Expectation 
with respect to P* is denoted by E*. 

For a subset A of E, denote by Ta (resp. Tjj") the hitting (resp. return) time of 
a set A: 

T A := inf {s >fl:I s £i}, 

T% := inf {t > : X t G A, X s ± X Q for some < s < t} . 

When the set A is a singleton {a}, we denote T[ a y, T^ a y by T a , T+, respectively. 
We set for every x in E, Mix) = fi(x)X(x). 
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Definition 2.1. For two disjoint subsets A, B of E, the capacity between A and 
B is defined as 

ca P (A, B) = J2 M{x) P* [T+ > T+] . (2.3) 

Clearly, the sum may be infinite. We prove below in Lemma [2.3l that the capacity 
is symmetric: cap(A, B) = c&p(B,A). 

We may also express the capacity in terms of the distribution of the adjoint 
process. By (|2.2[) . for any sequence Xq, x\, . . . , x n such that p(xi,Xi+\) > 0, < 
i < n, M(x )]J ^ l<n p{x l ,x l+1 ) = M(x n )H < i<n p*(x l+1 ,Xi). It follows from this 
observation that for any a G A, b G B, M(a)F a [T B <T^,T B = T b ] = M(b)¥* b [T A < 
T+,T A = T a ].Rence, 

cap(A, B) = M(a)F a [T B <T+] - M(a)P a [T B <T+,T B = T b ] , 

a£A a£Ab£B 

so that 

cap(AB) = 5>f(6)P fc *[T+ < T+] = C3l p*(B,A). (2.4) 

beB 

As in the reversible case, the capacity is a monotone function in each of its 
coordinates: 

Lemma 2.2. Fix two disjoint subsets A, B of E. Consider two sets A' , B' such 
that A C A' C B c and B C B' C A c . Then, 

cap(A,B) < c&p(A,B'), cap(A, J B) < cap(A',B) . 

Proof. The first claim follows from the original definition and the second one from 
equation (12. 4|) . □ 

For two disjoint subsets A, B of E, let Va,Bi VX b ■ E — > [0, 1] be the equilibrium 
potentials defined by 

V a ,b(x) = W X [T A < T B ] , VX, B (x) - V* X [T A < T B ] . (2.5) 

When the set B c is finite, the equilibrium potential V A . B has a finite support 
and belongs therefore to the domain of the generator. Moreover, in this case, V a ,b 
is the unique solution of the elliptic equation 

(LV)(z) = 0, z e E\(AUB) , 
V{x) = 1 , x G A , 
V(y) = , y G B . 

Furthermore, since by the Markov property, — (LV Ai b){x) = X(x)F x [T B < T^], 
x e A, 

ca P (A,B) = {V A . B , {-L)V a>b )„ = D{V A>B ) . (2.6) 

This identity does not hold in general, since the scalar product is not well defined 
if the set B c is not finite. However, following [2], if the process X t is positive 
recurrent and the measure M(x) — fi(x)\(x) is finite, one can show that this 
formula for the capacity holds. 

Lemma 2.3. For any disjoints subsets A and B of E, 

cap(A, B) = cap(B, A) . 
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Moreover, if {K n | n > 1} is an increasing sequence of finite sets such that E — 
U n >iK n , then 

cap(A, B) — lim lim cap(A m , B n ) , 
m— >+oo n— >+oo 

where A m =An K m , B n = B U K c n . 

Proof. Assume first that B c is finite. In this case by (|2.6|) . 

ca P (A,B) = D{V A ,b) = D(1-V b ,a) = D(V b ,a) ■ 

Since J2 X £B,yeB° ^{x)r{x, v) and J2 v eB,xeB" v( x ) r ( x , v) are nnite ; and since v b,a 
is equal to 1 on B, we may write D(Vb,a) as 

\Y fJ>(x)r(x,y)V B ,A(x)(VB,A(x) - V B . A {y)) 

+ ^M(2/M2/,a;)^B,A(y)(^(2/)-V B ,A(x)) (2.7) 
x,y 

+ J2 c 4 x ,y)VB,A(y)(.VB,A(y) - v B . A ( x )) , 

x,y 

where c a (x,y) = (l/2)[[i(x)r(x, y) — n(y)r(y,x)] and all these sums are absolutely 
convergent since Vb,a is bounded. 

The first two lines of the previous sum are equal. Since Vb,a is bounded, 
(LVb,a)(x) is well defined by (12.1[) for each x in E. Moreover, (LVb,a)(x) = 
for x e {AUB) C and -(LV B , A )(x) = X(x)F x [T A < T+], x e B. Therefore, the sum 
of the first two lines is equal to 

J2v( x )VbA x )(-LVb,a)(x) = ^M(fo)P 6 [T+ <T+] = ca P (S,A). (2.8) 

x beB 

On the other hand, since c a {x, y) = —c a (y, x) and since the sum J2 X y u ^,y ma y 
be written as (1/2) J2 X yi u x,y + u y,x}i the last line in (|2.7[) is equal to 

x,y 

= \H c a {x,y){Vi A {y)-ViA x )) + Y.T. c ^ x ^ l - V BA x )) 

x,y£B x£B yEB 

= - mi Ca ( x > v) v b,a( x ) + Ca & y) ■ 

xgByeE x£By£B 

As c a (a;,y) = —c a (y,x), J2x,y<gB c a( x iV) = 0- We may therefore replace the 
sum over B in the last term by a sum over E. Since fi is a stationary state, 
Syes c a( x ' 2/) = f° r au x E E. This proves that the last line of the previous 
displayed formula vanishes. In conclusion, when B c is finite, 

cap(AB) = D{Vb,a) = cap(S,A). 

It remains to remove the assumption that B c is finite. Let {K n \ n > 1} be an 
increasing sequence of finite sets such that E = U n >iK n . For each m < n, let 
A m — An K m , B n = B U K° and note that B^ is finite for each n > 1. Since each 
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set A m is finite, by (|2.3p . by (|2.4j) and by Beppo Levi's theorem, 

lim lim cap(A m ,i? n ) = lim ca,j>(A m ,B) = lim csq)*(B,A m ) 

tit— >+oo n— >+oo ' m— >+oo m— >+oo 

= lim V M(6) P* b \T+ > Tt } = cap*(B,A) = cap(A,B) . 

beB 

Since B^ is finite, by (|2.4[) and by the first part of the proof, for any m < n, 
cap(A m , £?„) = cap(i? n , A rn ) = cap*(A m , B n ). Repeating the same computations 
we obtain that 

lim lim cap(A m ,i? n ) — lim lim cap*(A m , B n ) — cap(B, A) . 
This proves the lemma. □ 

Denote by S (resp. A) the symmetric (resp. anti-symmetric) part of the genera- 
tor L in L 2 (fi): S = (1/2){L + L*}, A = (1/2){L- L*}. The next result is proved 
in Section [3] 

Theorem 2.4. Fix two disjoint subsets A, B of E, with B c finite. Then, 

cap(A, B) = inf sup h{L*F , fl) M - (ZT, (-S)i/) M } , (2.9) 

where the swpremum is carried over all functions H : E — > R which are constant 
at A and B , and where the infimum is carried over all functions F which are equal 
to 1 at A and at B. Moreover, the function Fa.b which solves the variational 
problem for the capacity is equal to {\/2){Va,b + B }, where Va,b, Vt b are the 
harmonic functions defined in (|2.5I) . 

In the reversible case, the supremum over H in the statement of Theorem 12.41 is 
easily shown to be equal to {(—L)F , F)^ and we recover the well known variational 
formula for the capacity: 

cap(A,B) = inf((-L)F, f% , 

F 

where the infimum is carried over all functions F which are equal to 1 at A and 
at B. 

When the set E is finite and the sets A and B are singletons, A — {a}, B = {&}, 
the supremum over H becomes a supremum over all functions H : E — > K. In this 
case, 

sup \2{L*F, B)„ - {H,(-S)H) M } = (L*F, (S)- X L*F)^ . (2.10) 

Therefore, when the set E is finite, since L (S^L* = {[(-L) _1 ] s } _1 Q21 Section 
2.5], the formula for the capacity between two singletons becomes 

cap({a},{&}) = inf (F , Li-S^L^F), = inf (F , {{(-Ly^F), , 

r r 

where the infimum is carried over all functions F which are equal to 1 at a and 
at b, and where [(— L) _1 ] s stands for the symmetric part of the operator (— L)~ l . 

In Lemma 14.11 below we express the right hand side of (|2 . 10[) as an infimum over 
divergence free flows. We have therefore two alternative formulas for the scalar 
product (L*F, (—S)~ 1 L*F) fM , one expressed as a supremum over functions, and 
another one written as an infimum over flows. 
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2.1. Estimates on the capacity. We compare in this subsection the capacity 
associated to the generator L with the symmetric capacities cap s associated to the 
generators S. Let {X^ : t > 0} be the Markov process on E with generator S. We 
shall refer to as the symmetric or reversible version of the process X t . Denote 
by P*, x £ E, the probability measure on the path space D(M.+ , E) induced by the 
Markov process X* starting from x. 

For two disjoint subsets A, B of E, let cap s (^4, B) be the capacity between the 
sets A and B for the reversible process X*: 

cap'(A,S) = ]TA/(.*)P*[T+ >T+] . 
xeA 

In the case where the set B c is finite, 

Ca p s (A,B) = {VX,B,{S)VX >B )^ 

where VX b IS tne equilibrium potential: VjJ B {x) — P* [Tji < T B }. Moreover, since 
the generator S is symmetric in L 2 (/i), it is well known that if B c is finite, 

cap s (A,B) = inf ((—S)F , F)^ = M {(-L)F , F}^ , (2.11) 

f F 

where the infimum is carried over all functions F which are equal to 1 at A and 
at B. Taking H — —F in the variational formula (|2.9j) we obtain that 

cap(A,B) > inf((-L)F, F)^ . 

F 

The next result follows from the previous observation, (|2.11j) and Lemma [2~3l 

Lemma 2.5. For two disjoint subsets A, B of E, 

ca,p s (A,B) < cap(A,B) . 

Recall that a generator L satisfies a sector condition with constant Cq if for every 
/, g in the domain of the generator, 

(Lf,g)l < C ((—L)f, f) M ((—L)g, g)^ . 

Next result, whose proof is presented at the end of Section G2 shows that if the 
generator L satisfies a sector condition, we may estimate the capacity between two 
sets by the capacity associated to the symmetric part of the generator. 

Lemma 2.6. Suppose that the generator L satisfies a sector condition with constant 
Cq. Then, for every pair of disjoint subsets A, B of E, 

cap(A,B) < C* cap s (A,B) . 

2.2. Flows in finite state spaces. We have seen that one can reduce capacity 
computations to the case when B c is finite. By identifying B with a single point 
(this will be rigorously done in the next section) we can then restrict ourselves to the 
case of a finite space E. We then assume in this subsection that the E is finite. In 
this case the stationary measure fi is unique up to multiplicative constants. Define 
the (generally asymmetric) conductances 

c(x,y) = n{x)r(x,y) , c*{x,y) = fi(x) r*(x, y) , x^yeE. 

Note that c(x, y) = c*(y, x). Let c s (x, y), c a {x, y), be the symmetric and the asym- 
metric parts of the conductances: 

c s (x,y) = (l/2){c(x,y) + c*(x,y)} , c a (x,y) = (l/2){c(x,y) -c*(x,y)} , 
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for x y G E. Clearly, c s (x,y) = c s (y,x) and c a {x,y) = -c a (y,x). The sym- 
metric conductances c s (x, y) are the conductances of the reversible Markov process 
associated to the generator S. 

Denote by £ the set of oriented edges or arcs of E: £ — {(x,y) e E x E : 
c s (x, y) > 0}. For an oriented edge e = (x, y) G £ , let e~ = x be the tail of the arc 
e and let e + = y be its head. We call Row any anti-symmetric function ip : £ — > K. 
Denote by T the set of flows endowed with the scalar product 

M) = \ Y, —nr^<feiv)1>fav) > ( 2 - 12 ) 

and let || • || be the norm associated to this scalar product. 

Denote by (div ip)(x), x G E, the divergence of the flow ip at x: 

(div >p){x) = Y ¥>( x >y) ' x e ^ • 

y:(x,y)e£ 

A flow </5 whose divergence vanishes at all sites, (div(^)(a;) = for all x G E, is 
called a divergence free flow. An important example of such a divergence free flow 
is c a . 

For a function / : — > R, let *£f(x,y) = c s (x,y)[f(x) — /(y)] be the gradient 
flow associated to /. Clearly, ^/ belongs to T and 

ll*/H 2 = (*/,*/) = <(-£)/> />*■ (2-13) 

Let C/ = {\l//|/:i?— > 1} C J. We refer to £/ as the set of gradient flows. 

A finite sequence 7 = (xq, ■ ■ ■ , x n — xq) of sites in i? which starts and ends at the 
same site is called a cycle if (xi, Xi+i) is an arc for each < i < n — 1 and if xi 7^ 
for < i < j < n. An arc e is said to belong to a cycle 7 = (xo, . . . , x n — xq) if 
e = [x,i,Xi + i) for some < i < n. We associate to a cycle 7 = [xq, . . . , x n = xq) 
the flow x 7 : £ — > Ml defined by 

n-1 

Xi = X^^i.^+O -< Wi,*i)} • (2-14) 

i=0 

Denote by C the subspace of J 7 spanned by flows associated to cycles. 
A flow if, > G C associated to a cycle has no divergence: 

(div (p)(x) — x e E . 

Also, 99 is a gradient flow if and only if ip is orthogonal to all cycle flows. In other 
words, we have 

f = g®c, g±e. (2.15) 

In addition, a flow ip that is orthogonal to all gradient flows satisfies, (divip)(xo) = 
for all xq in E, because (divyj)(xo) = (^Sf,ip) for the function / defined by f(x) — 
5 Xo ,x- This proves that C is the set of all divergence free flows. 

Inspired by the computation of the current through an arc (x,y), presented in 
(|4.5|) below, for a function / : E — > M, denote by $/ : £ — s- R the flow defined by 

= f(x)c(x,y) - f(y)c(y,x) . (2.16) 

In Section |4] we prove the following result. 
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Theorem 2.7. For any disjoint and non-empty sets A, B d E , 

cap(^,i3) = inf inf ||$/ - ip\\ 2 , 

/ v 

where the first infimum is carried over all functions f : E — > M which are equal to 
1 on the set A and on the set B, and the second infimum is carried over all flows 
ip G T such that 

(div <p)(x) = , xe(AuB) c , 5^(divp)(a)=0, ^(div <p)(b) = . 

a£A beB 

In the case where A and £ are singletons, the second infimum is carried over all 
divergence free flows. Hence, in the case of singletons, the infimum corresponds to 
a projection over the space of gradient flows. 

In the last section of this article, when we shall estimate some capacities among 
singletons, the divergence free flow ip{x,y) = c a (x,y), (x,y) € E, will be used 
repeatedly to obtain upper bounds. 

2.3. Transient Markov processes. Assume in this subsection that the irre- 
ducible Markov process {X t \ t > 0} is transient, and denote by G(x,y) its Green 
function: 



G(x,y) = E x / l{X t = y}dt . 

l Jo J 
Define the capacity of a state x £ E, denoted by cap(x), as 

cap(x) = M (x) P x [T+ = oo] . 

Since G(x, x)^ 1 = A(a;)P x [T+ = oo], we have that 

cap(» = fi(x) — • (2.17) 
G(x, x) 

Fix a finitely supported function / : E — > M. such that f(x) ^ 0, and let F(y) = 
f(y)/f(x) so that F(x) = 1. By Definition 12. 1[ if {A n \ n > 1} is a sequence of 
increasing, finite sets such that E = \J n >iA n , 

cap(ir) = lim c&p(x,An) ■ 

n—>oc 

Since A n is finite and since F is finitely supported, with F(x) = 1, by Theorem l2.4[ 
cap(x) < lim sup h(L*F , H)„ - {H,(-S)H)A , 

where Q3„ is the set of functions H : E — >• K which vanish at A c n . As F( ■ ) 
/: ■ )/f(x), replacing H by H'( ■ ) = H ( • )/f(x), we obtain that 

cap(x) < lim sup {2(L*/, - <^(-^)/.| ■ 

/(a;) 2 n-voo HeQSn I J 

In view of ()2.17j) . we have proved the following result, which generalizes a well 
known estimate in the context of reversible Markov processes [T2] Proposition 5.23], 
[1] Lemma 2.1]. 

Lemma 2.8. Let f : E — > M be a finitely supported function and let {A n \ n > 1} 

&e a sequence of increasing, finite sets such that E — U n >iA n . Then, for every 
x e E, 

fi(x)f(x) 2 < G(x,x) lim sup \2(L*f,H)^ - (H,(-S)H)A , 
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where Q3 n is the set of functions H : E — > R which vanish at A c n . 

3. Collapsed Chains and Proof of Theorem 12.41 

We start this section by assuming that E is finite and that /i is the unique 
stationary probability measure. In the case where the sets A and B are singletons, 
the proof of Theorem 12.41 takes the following form. 

Lemma 3.1. Fix a pair of points a ^ b in a finite set E. Then, 

cap({a}, {&}) - inf (/ , L (-S)"^* j% , (3.1) 

where the infimum is carried over all function f : E M such that f{a) = 1, 
f(b) = 0. Moreover, the function f ab which solves the variational problem (|3.1[) is 
unique and equal to (l/2){V^.b + V* b }, where V a ^, V* h are the harmonic functions 
defined in (|2.5|) . 

Proof. The operator L(— £?) _:L Z* restricted to the space of mean zero functions is 
symmetric, strictly positive and bounded because the state space is finite. There 
exists, in particular, a unique function f a ^ which solves the variational problem 
(|3.ip . Moreover, as (— S)^ 1 is also strictly positive on the the space of mean zero 
functions, there exists a strictly positive constant Co such that 

{fa,b,L{-S)- 1 L*f afi )^ > C Q {L*f afi ,L*f afi )^ > 0. (3.2) 

The previous expression can not vanish due to the boundary conditions of f a ,b- 
Since f a , b solves the variational problem (|3.1I) . 

{L{-S)- l L*f a , b ){x) = 0, x^a,b. 

Let W a ^b = S~ 1 L*f a .b + c , where Co is a constant chosen for W a ,b to vanish at b: 
W a ,b(b) — 0. Since LW a ,b = on E \ {a,b}, W a ^ is a multiple of the harmonic 
function V a ,b introduced in (|2.5p : W a ,b = AV^t, where A = W a ,b{a). 

We claim that A = 1 so that W a ,b — V a ,b- Indeed, since W a ,b is harmonic on 
E\{a,b} and f a>b (a) = 1, f a , b (b) = 6, 

(f a ,b, L (-Sr^/aA = (fa.b , (-£) W a . b ) ^ = -/i(a) (L W a , b ) (a) . 
On the other hand, since W a ,b — Co = S L* f a>b and SS^ 1 is the identity, 

(f a , b , L(-5)- 1 L*/a,6> M = (L* f a , b , (S)- 1 L* f a , b ) „ = {W a , b , (S)W a ,b)» 
= (W a , b , (-L)W a , b )n = -»(a)W a , b (a){LW a , b )(a) , 

where the last identity follows from the fact that W a , b is harmonic on E \ {a, 6} and 
that W a , b (b) = 0. By (|3.2[) and the two previous displayed formulas, W a ,b(a) = 1 
so that W a ,b — V a ,b- Hence, by the last displayed formula and (|2.3[) . 

(f a<b , L(-5) _1 iVa,6) M = (-i)V r „,6) M = cap({a},{fe}) , 

which concludes the proof of the first assertion of the lemma. 

Denote by f a ^ b a function which solves the variational problem (I3.ip . We claim 
that f a ^ = (1/2){V Q ,6 + V* b }. Indeed, since W a ,b = V a , b and since V a , b is L- 
harmonic on£\ {a,b}, on this set (l/2)L*V a , b = SV a ,b = L*fa,b- Furthermore, as 
V* b is L*-harmonic on E\{a, b}, we have in fact that (1/2)L* {V a<b +V* b } = L* f a , b . 
Hence, 

L*(l/2){V a , b + V£ b } = L*f a , b OTiE\{a,b} 
and {l/2){V a , b + V* b } = f a , b on {a,b} . 
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It follows from these two identities that f a .b = (l/2){V a ,b + V* b }. □ 

To extend the previous result to the case where the sets A and B are not sin- 
gletons, we define a Markov chain in which a set is collapsed to a single state. Fix 

a subset A of E, and let E A = [E\A\\J {£)}, where is an extra site added to E 

A 

to represent the collapsed set A. Denote by {X t : t > 0} the chain obtained from 
X t by collapsing the set A to a singleton. This is the Markov process on Ea with 
jump rates r A (x,y), i,j/6 E A , given by 

rA{x,y) = r(x,y) , ¥a(x,D) = }] r(x, z) , x,y£E\A, 

1 _ ( 3 - 3 ) 
r A (Q,x) = —7-^2^,n{y)r{y,x), xtE\A. 

Ml > yeA 

A 

The collapsed chain {X t : t > 0} inherits the irrcducibility from the original chain. 
Denote by JIa the probability measure on Ea given by 

jL A (d) = fi(A) , JJ A (x) = n(x) , x£E\A. (3.4) 

Since 

c (y> z ) = c ( z '^) ' 

y£A,z£A y£A,zeA 

one checks that ~fi A is a stationary state, and therefore the unique invariant proba- 

A 

bility measure, for the collapsed chain X t . 

We may extend the concept of collapsed chain to the case in which more than 
one set is collapsed to a singleton. One can proceed recursively, collapsing first 
a set A to a point a E, obtaining a Markov chain in (E \ A) U {a}, and then 
collapsing a set B C E, BD A — 0, to a point b $ EU{a}, obtaining a new Markov 
chain in [E \(A\J B)] U {a, b}. One checks that the final process is the same if we 
first collapse B and then A. The rate Ta,b{o-, b) at which the collapsed chain jumps 
from a to b is given by 

r A ,B(a,b) = -tttX^^H r ( z > x ) ■ ( 3 - 5 ) 
^ A > zeA xeb 

Denote by La the generator of the chain {X t : t > 0} and by L A the adjoint 

of L A in L 2 {Jl a ). Recall that we represent by {X t * : t > 0} the adjoint of the chain 

A 

Xt and by L* its generator. Let {X* t : t > 0} be the chain obtained from XI by 
collapsing the set A to a singleton and by L* A the generator of this process. We 
claim that 

La* = ~L~* a ■ (3.6) 

A 

To prove this claim, denote by r^(x 1 y) the rates of the adjoint of {X t : t > 0}: 

MaM^) ) e 

Let r*(x, y), x, y € be the jump rates of the adjoint process and let r* A (x,y), 
x, y € be the jump rates of its collapsed version. 

In view of the previous displayed formula and by (|3.3[) . (|3.4p . for y G _E \ A, 

_*/ \ v(y)r(y,x) — . . 

^A ^.y) = n — = r (y^ = ?-*A a ; >2/)- 
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Furthermore, for y G E \ A, since H A (J>) = n(A), by 



n(y)r A {y,D) _ v(y)J2 z &A r (y, z ) 



»(A) 11(A) 



Finally, for x S E \ A, by analogous reasons, 

_*/ ^ n{A)T A (p,x) T,zzAt J '( z ) r ( z > x ) 
r A (x,d) — = 

= E r *( x > z ) = r*A(x,Q) , 
zeA 

which proves p.6[) . It follows from this result that 

S A = (1/2){L A + L* A } , (3.7) 

if S A stands for the generator S = (1/2)(L + L*) collapsed on the set A. 

Fix two functions /, g : E A -)• R. Let F, G : E -> R be defined by = /(at), 
F(z) = z £ A, with a similar definition for G. We claim that 

(LAf,9h A = (LF,G)^. (3.8) 
Conversely, if F, G : E — > R are two functions constant over A, (|3.8I) holds if we 
define /, g : E A -» R by /(z) = F(x), /(?>)_= F(*) for some zeA 

Fix two functions /, g : E A — > R. By definition of L A , 

(L A f,g)-p A = E l J A (x)r A (x,y)[f(y) ~ f(x)]g(x) . 

x,v£Ea 

In view of (|3.3[) . (|3.4[) . this expression is equal to 

E E ^y)[/(y)-/w] + E r ^ z )[/w-/( a; )]}-9( a; ) 

cceB\A y£E\A zGA 

+ E £M(*)r(z,y)[/(y)-/(a)]<tf>). 

y£E\A zGA 

Since F(x) = f(x) for x G F\ A, and F(y) = for yei, with similar identities 
with G, g replacing F, /, the last sum is equal to 

E E ^,y)™-^)] + E r ( a; ' z )^( z )-^( :z; )]} G ( a; ) 

xG-E\A y£E\A z6i 

+ E E »(z)r(z,y)[F(y)-F(z)]G(z). 

zGA y£E\A 

Since F is constant on A, we may add to this expression 

EE^W^I/) [F(y)-F(x)}G(x) 

x£A y&A 

to obtain that the last displayed expression is equal to (LF,G)^, which concludes 
the proof of the first assertion of (|3.8[) . The second statement is obtained following 
the computation in reverse order. 
It follows from ((Ml) and (J^i that 

(p A f , g)^ = (L*F , G) M , (S A f , g)^ = (SF , G>„ • (3-9) 
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The next assertion establishes the relation between collapsed chains and capac- 
ities. Fix two disjoint subsets A and B of E. Let E a,b = [E\(A\J B)] U {a, 6}, 

A B 

where a ^ b are states which do not belong to E. Denote by {X t : : t > 0} the 
chain in which the sets A, B have been collapsed to the states a, b. Let Ta,b(x, y), 
Pa b(. x tV)j an d Aa,b(x), x, y G Ea.b, be the jump rates, the jump probabilities, 

A B 

and the holding rates, respectively, of the chain X t ' , and denote by JL A b its 
unique invariant probability measure. 

Denote by cap^ B the capacity associated to the collapsed chain. We claim that 

5apA,B(W,W) = cap(A,B). (3.10) 

A q 

Denote by ' , x £ Ea,b, the probability measure on D(R +1 Ea,b) induced by 

A 

the collapsed chain X t ' starting from x. By Definition 12. 1[ 

SSPa,b({o}.W) = MA, B (a)Pa' B [T+>T+] 

= 37^(0) J! PA, B {^ x )^t B [Ta>T b } , 

whereMA iB (i) = /I AjB (x)Aa,.b(2;)- Since M J 4 )S (a)p y4 B (a,a;) = 7Z AjB (a) r A ,B{a,x) 
and since B (a, a ) — 0, by the explicit expression (|3.3[) for the rates of the col- 
lapsed chain, the previous expression is equal to 

Ma,bO) — r^r5Z^( z ) r ( z ' x )^' S [ Ta >Tb ] 

+ l I A,B( a ) 7 A,B(a,b)Ff' B [T a >T b ] . 

By construction, P^' B [T > T b ] =¥ X [T A > T B ] for x G £7\[AUB], andPf' B [T a > 
T 6 ] = 1 = P^Ta > Tb], x G B. Hence, as /z^4 — by (|3.5|) the last sum 

is equal to 

J2 Y,n(z)r(z,x)F x [T A >T B ] . 

x£E\A z£A 

Since P^Ta > Tg] = 0, x G A, and since y,(z)r(z,x) — M(z)p(z,x), this expres- 
sion is equal to 

= ]TA/(z)P 2 [T+>T+] = cap(AS), 

zeA 

which concludes the proof of claim (|3.10|) . 

Lemma 3.2. Fix two disjoint subsets A, B of a finite set E. Then. 

c& P (A,B) = inf sup h{L*F, H)„ - {H,{-S)H)A . 

F H L J 

where the supremum is carried over all functions H : E — > R which are constant 
at A and B , and where the infimum is carried over all functions F which are equal 
to 1 at A and at B. Moreover, the function Fa.b which solves the variational 
problem for the capacity is equal to (\/2){Va,b + V A s} ; where Va,b, V a b are ^ e 
harmonic functions defined in (|2.5[) . 



DIRICHLET PRINCIPLE FOR NON REVERSIBLE MARKOV CHAINS 



13 



Proof. Fix two disjoint subsets A, B of E. By Lemma \3. II and identity (|3.10p . 

cap(AB) = inf (Ll, B f, (Sy'T^f)^ , (3.11) 

A B 

where La.b is the generator of the chain {X t ' : t > 0} introduced right after 
(|3.9[) . 5 is the symmetric part of La,b, S = (1/2)(La,b + L A b)> and where the 
infimum is carried over all function / : E A>B K such that f(a) — 1, /(&) = 0. 

By the variational formula for the norm induced by the operator (— S)^ 1 , the 
previous expression is equal to 

inf sup \2(L* AB f , % - (h, (-<S)% } , 

where the supremum is carried over all functions h : E a.b K- By (|3.6I) . L A B — 
L*a,b, and by (|3.7p . S — Sa,b, where Sa, b is the generator S collapsed at A and 
B. Hence, the previous displayed equation is equal to 

inf sup {2(1^/, % A fi - (M-^, B )% A B } . 

Let F, H : E -> K be defined by F(ar) = /(sc), x £ £\(iU B), F(z) = f(a), 
z eA, F(y) = f(b), y e B, with a similar definition for H. By ([375]) . (j3Tg]> . the last 
variational problem can be rewritten as 

inf sup |2(L*F , - (H, (-5)fl) M | . (3.12) 

F H l ) 

where the supremum is carried over all functions H : E — > K which are constant at 
A and -B, and where the infimum is carried over all functions F which are equal to 
1 at A and at B. This proves the first assertion of the lemma. 

To prove the second assertion of the lemma, recall from Lemma 13.11 that 

cap(A,B) = (L A B f a , b , {-SY 1 L* AB f atb )-p; AB , 

where f a ,b — (l/2){V a ,b + V a b } and V a ,b, V a b are the harmonic functions for the 
collapsed process. By the first part of the proof, the right hand side is equal to 

sup{2(L*F A , B , #)„ - (H,{-S)H)^ , 

where the supremum is carried over all functions H : E — >• R which are constant 
at A and B, and where Fa,b(x) = faA x )> x e E\(Al) B), Fa,b{z) = 1, z e A, 
Fa,b{v) = 0, y E B. As we have already seen, by construction of the collapsed 
process, for x G E \ (A U B), 

V a>b (x) = vt' B [T a <T b ] = V X [T A <T B ] - V a ,b{x), 

with a similar identity for V a b . In conclusion, 

cap(A-B) = sup {2(L*F a ,b , H> M - (ff, , 

where i^.s = (1/2){Va,b + VI B }, concluding the proof of the lemma. □ 

Remark 3.3. The expression inside braces in the displayed formula of Lemma \3.£\ 
does not change if H is replaced by H + c, where c is a constant. We may therefore 
restrict the supremum to functions H which vanish at B. 
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We finally turn to the case where E is denumerable. Fix two disjoint subsets A, 
B of E and suppose that B c D A is finite. Similarly to what we did earlier in this 
section, we define chain where the infinite set B is collapsed to a state. 

Denote by X t the Markov process on the finite set B c U {D}, where D is an extra 
site added to E to represent the collapsed, possibly infinite, set B, whose rates 
r(x,y), x, y £ B c U {D}, are defined by 

r(x,y) = r(x,y) , r(x,D) = ^r{x,z) , x,y £ B c , 

_ Z£B (3-13) 

r(D,x) = 2J Ky)r(y,x) , x £ B c . 
y£B 

Note that r(D,x) is finite because J2 v ee f^iv) r (v^ x ) ~ M(x) < °°! as /i is a sta- 
tionary state, and that r($,x) > if there exists z £ B such that r(z,x) > 0. In 
particular, the collapsed chain {X t : t > 0} inherits the irreducibility from the orig- 
inal chain. Moreover, since Exes- J2 y eB t*(v) r (y> x ) = J2xeB J2 y eB- M?/M2A %), 
Ji{x) = fJ,(x), x £ B c , ~p(d) = 1 is a stationary measure. 

Let Pa, a; 6 £? c U {D}, represent the probability measure on the path space 
D(M. + ,B C U {D}) induced by the Markov process X t starting from x. Clearly, for 
any A C B c , 

P y [T B < T+] = P„ [n < T+] , V G B c . 
Therefore, by for any A C F c , 

cap(A,B) = ^M(y)P. y [T B <T+] = £ M(y) P„ [T» < T+] - cap(A,D) , 

2/£A yeA 
_ (3 - 14) 

if cap stands for the capacity of the collapsed chain. 

Denote by L the generator of the collapsed chain. Fix a pair of functions /, 
h : B c U {D} -> R such that /j(D) =0. Let F, : E -» R be the functions defined 
by F(x) = /(x), x G -B c , F(z) = /(D), z £ B, with a similar definition for H. We 
claim that 

(L/,% = (LF,H)„ . (3.15) 
Conversely, if F, H : E — > R are constant on the set J3 and if H vanishes at B, 
(|3~T5| holds if /, h : F C U{D} -J- R are defined by /(a;) = F(x), x £ B c , /(D) = F(z), 
z £ B, with a similar definition for h. 

To prove (|3.15[) . fix a pair of functions /, h : B c U {D} — > R with the above 
properties. By definition of the collapsed chain and since h(d) — 0, (Lf, h)-p is 
equal to 

n(x)r(x,y)h(x)[f(y)-f(x)}+ ]T »(x)r(x,D) h(x) [/(D) - /(*)] • 

x,yeB c xeB c 

Since r(x,D) = X^eB r ( x ; z ) an( i sm ce F is constant over B, the second term is 
equal to 

J2 ^Kx)r(x,y)h(x)[m-f(x)} = £ £ M (x) r(x, y) H(x) [F{y)-F{x)\ . 

x£B a yeB xeB c yeB 

Hence, adding the two terms, 

(!/,% = ^2v(x)r(x,y)H(x)[F(y)-F(x)} = (LF, H)^ 

xeB a yEE 
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because H vanishes on B. This proves the first assertion of claim. The converse 
one is proved by following the previous computation in the reverse order. 

Proof of Theorcm \2.4\ Fix two disjoint subsets A, B of E and assume that B c is 
finite. By (|3.14j) . cap(A, B) — cap(A,Q). On the other hand, and by Lemma I3~2l 
and by Remark 13. 31 

cEp(A,D) = inf sup h(f , Lh)^ - (h, (-1)%) , 

/ h 1 > 

where the supremum is carried over all functions h : B c U {0} — > R which are 
constant at A and vanish at D, and where the infimum is carried over all functions 
/ which are equal to 1 at A and at d. Since / vanishes at 0, by claim (|3.15[) . the 
right hand side of the previous is equal to 

inf sup \2(L*F, - (H,(-L)H)A 

F H I J 

where the supremum is carried over all functions H : E — > K which are constant at 
A and vanish at B, and where the infimum is carried over all functions F which are 
equal to 1 at A and at B. The expression inside braces in the previous formula 
remains unchanged if we replace H by H + c, where c is a constant. We may 
therefore veil the assumption that H vanishes at B. This proves the first assertion 
of Theorem E3 

By (|3~T4l) and by Lemma EH) 

cap(A,B) = cSp(A,d) = sup |2(/ j4 ,d, Lh)^ - (h, , 

where J^o = (1/2){Va,b + ^A,a}i an d Va,t>, V^ d are the harmonic functions asso- 
ciated to the collapsed process and to its adjoint. By (|3.15l) . 

cap(A,B) = sup{2(i*F A , B , H) M - (H,(-L)H)^ , 

where Fa : b{x) = fA,d(x), x G B c , Fa.b{ z ) — 0, z £ B. By construction of the 
collapsed process, Va,d = Va,b and V\ = VX b on where Va,b and VX b are 
the harmonic functions of the original process. □ 

Proof of Lemma 12.61 Fix two disjoint subsets A, B of E and assume that B c is 
finite. By Theorem 12.41 the capacity cap(A, B) is given by (|3.12l) . By the sector 
condition, the expression inside braces in this formula is bounded by 

2y/C Q {{-S)F , F) l J 2 {{-S)H, H) 1 ' 2 (H,(-S)H), . 

The supremum over H is thus bounded by Co((—S)F , F)^. Therefore, 

cMAB) < C inf((-5)F,^) M , 

r 

where the infimum is carried over all functions F equal to 1 at A and at B. 
By definition of the capacity in the reversible case, the right hand side is equal to 
CoC&p s (A, B). This proves the lemma in the case where the set B c is finite. To 
extend it to the general case, it remains to apply Lemma l2~3l □ 
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4. Flows and Proof of Theorem 12.71 

We assume in this section that the state space E is finite. We first prove Theorem 
I2.7l in the case where the sets A and B are singletons. The proof relies on an identity, 
established in Lemma |4. II below, which provides a variational formula for the norm 

(fAK-L)- 1 }*}- 1 /)}' 2 . 

Before stating this result, we start with an elementary observation. We claim 
that 

two gradient flows ^f, ^ g are equal if and only if / — g is constant . (4.1) 

Indeed, if the gradient flows are equal, since ^/ — ^ g = &f-g, in view of (|2 . 1 3[) . 
((—L)(f — g), (/ — g))^ — which implies that / — g is constant. The converse is 
obvious. 

Recall that we denote by C the set of divergence free flows and by $/ the flow 
associated to a function / : E — >• R introduced in (|2.16j) . 

Lemma 4.1. For every junction f : E — > R, 

(fAK-L)- 1 }*}- 1 !), = (L^fASr'L*/), = ini :||*/-^| a . 

Proof. Fix a function / : E — >■ R. Since $/ is a flow, by ()2.15p and by (|4.1|) there is 
a function W : E — > R, unique up to an additive constant, and a unique divergence 
free flow A/ such that 

$/ = + A f . 

Computing the divergences of each flow we obtain that L* f — SW so that W = 
S~ l L* f + Co for some constant cq. Therefore, since = ^w+c for any constant 
c, $v, with V = S~ 1 L*f, is the the projection of the flow on the space of 
gradient flows. Moreover, by (|2.13[) . 

(*v,*v> = (V,(S)V)„ = (L*f, (-5)- x i*/>^ (4.2) 

because (L* /, 1) M = as fi is invariant. Furthermore, since is the projection of 
the flow <!>/ on the space of gradient flows, 

(#y,*v) = inf ®f~<p) , 

which concludes the proof of the lemma. □ 
Lemma 4.2. Fix a pair of points a =^ b in E. Then, 

cap({a},{6}) = inf ini : ||$/ - <p\\ 2 , 

where the infLmum is carried over all functions f : E — > R such that /(a) = 1. 
f{b) = 0. Moreover, the infimum is uniquely attained at 

f = (1/2){K,6 + K%} , <P = (V2) {*v. % - $v atb } , (4-3) 

provided for a function g : E —> R we denote by <£>* i/ie /7ow given by <fr*(x,y) — 
g(x)c*(x,y) - g(y)c*(y,x). 

Proof. The first assertion of the lemma follows from Lemmas 13.11 and 14.11 More- 
over, the function / which solves the variational problem for the capacity coincides 
with the one which solves the variational problem (|3.ip . Hence, by Lemma [3.11 
(l/2){V a ,b + V* b } is the unique functions which attains the minimum. It remains 
to show that (1/2) {$y» b — Q'y b } is the optimal divergence free flow. 
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Let F = (l/2){K,b + V* b }. We claim that (L*F)(x) = (SV a , b )(x) for all x e E. 
For x =/= a, b, this identity is obvious and has been derived in the proof of Lemma 
ELU For x = a, it reduces to the identity P*[T+ < T+] = P a [T+ < T+] which, in 
view of (|2.3|) . is equivalent to cap({a}, {&}) = cap*({a}, {&}). Since this identity is 
the content of Lemma l2.3l and since the same argument applies to x — b, the claim 
is in force. In particular, by the proof of Lemma |4. 11 tyy a b is the projection of the 
flow on the space of gradient flows, and there is a unique divergence free flow 
Ap such that 

$F = *V„ 6 + , (&f — A-f , ®f — A_f) = inf (& F — (p, $p — <p) , 

An elementary computations shows that Ap = $>p — \&y a b — (l/2){ < i ) y* — $y }, 
which completes the proof of the lemma. □ 

We may restate the previous lemma to obtain a variational formula for the 
capacity in terms of the Dirichlet form. 

Lemma 4.3. Fix a pair of points a ^ b in E. Then, 

cap({a},{6}) - MD{V) , 

where the infimum is carried over all functions V : E — > K. such that \l/y is the 
orthogonal projection on the space of gradient flows of some flow with f(a) = 1, 
f(b) = 0. Moreover, the infimum is uniquely attained, up to additive constants, at 

v = v a , b . 

Proof. By Lemma 13. 11 

cap({a},{&}) = inf<L*/, (-S)" 1 ^/^ , 

where the infimum is carried over all functions / : E — > R such that /(a) = 1, 
f(b) = 0. To conclude the proof of the first assertion of the lemma it remains to 
recall identity (|4~2j) . 

To prove uniqueness of V a ,b, recall from the proof of Lemma [4.21 that SV a ,b = 
L*F, where F = (l/2){V a ,b + V* b }. Hence, by the proof of Lemma Ol ^ Va b is the 
orthogonal projection of $f- Therefore, D(V a .b) > infy D(V). On the other hand, 
by (|4.2I) and since by Lemma I3TT1 F is the optimal function, D(V a .b) < infy D(V). 
This shows that V a , b is optimal. 

To prove uniqueness, suppose that W is another optimal function, and that 
is the orthogonal projection on the space of gradient flows of some flow $ s with 
g(a) = 1, g(b) = 0. By the optimality of W and by (|4~2j) 

cap({a},{6}) = D(W) = (L* g , (- S)' 1 L* g) „ . 

Hence, by the uniqueness of Lemma 13 - 1 1 g = F, and by the proof of Lemma 14. 1[ 
L*F = SW. Since L*F is also equal to SV a , b , we obtain that SV a ,b = SW, which 
implies that V a ,b — W is constant, as claimed. □ 

The flows $y* b and <fry which appear in the previous lemma have a simple 
probabilistic interpretation. Denote by {X„ : n > 0} the discrete time skeleton of 
the chain, and recall that M(x) = pi{x)\{x) is a stationary state for X„, unique 
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up to a multiplicative constant. For B C E, let Gb be the Green function of the 
process killed at B: 

Tfl-1 

G B (x,y) :=E X [J2 H*n = y}\ , 

where Tb (resp. t b ) stands for the hitting time of (resp. return time to) B for the 
discrete time chain X n : 

tb = min{n > : X n 6 B} , t b = min{n > 1 : X n G B} . 

In the same way, G* B , B C E, stands for the Green function of the time reversed 
chain killed at B. 

Denote by P x , x € E, the probability on path space D{T, + : E) induced by the 
Markov chain {X„ : n > 0} starting from x, and by 8 n , n > 0, the time shift by n 
units of time. Fix two disjoint subsets A, B of E. By the last exit decomposition, 
for every x E E, 

V x [ta < t b ] = X! P 4 X » eA,n<T B ,T+o6 n < r\ o 6 n ] 

n>0 

= J2Y. F * t X » = a,n<r B ] P a [t$ < r+] 

aGA n>0 

- Y / G B (x,a)¥ a [r+ <r+] . 

Since M{x)Gb{x, y) — M(y)G* B (y,x), it follows from the previous identity 

V A , B (x) = P x [r A <t b ] = ]T G *b (o, M(a) P a [r+ < r+] . (4.4) 

aeA ^ ' 

Denote by va,b the harmonic measure, also called the normalized charge distri- 
bution, 

v A ,B(a) = ^^ M ^ a [^<ri]. 

Fix two disjoint subsets A, B of E. Denote by i(x,y) = iA,B(x,y) the current 
through the arc (x, y) for the process which starts from the harmonic measure va,b 
and which is killed at B: 

Tfl-l 

i(x,y) := E VA B I ^ {H^n = x,X n+ i = y} - 1{X„ = y,X n+1 = x}} . 

n=0 

By the Markov property and in view of (|4.4I) . if we denote by i*(x,y) the current 
through the arc (x,y) for the time reversed chain, 



i*(x,y) ■■= ^2fA,B(a){G B (a,x)p*(x,y) - G* B (a,y)p*(y,x)} 

aeA 

= J2"AAa){j^-sGB(a,x)c*(x,y) - ^-G B {a,y)c*{y,x)) (4-5) 

= cap(A,By 1 {VA,B(x)c*(x,y) - V A , B (y) c*(y,x)} . 

Since this last expression is equal to c&p(A, B)~ 1 <& Va (x, y), $y A is, up to the 
multiplicative constant cap(^4,i3), the current through the arc {x,y) for the time 
reversed Markov chain X* started from the harmonic measure va b and killed at B. 
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Analogously, $yj B is, up to the same multiplicative constant, the current through 
the arc (x, y) of the discrete time Markov chain X„ started from the harmonic 
measure v\ B and killed at B. 

Given a function / : E — >• M, we may write the flow $/ as$/ = \E'/ + Y/, where 
Tf is the flow given by 

T /(^y) = c a {x,y){f(x) + f(y)} . 

It turns out that the flows <3// and Tf are orthogonal: 

<*/> T /> = \ E ^T7~y 2/) T /( x > 2/) = o. (4.6) 
Indeed, by definition of the flows *S? f and T/, 

* — ' c s [x,y) 

(x,y)e£ y ' x.yeE 

Since c a (x,y) = (l/2){c(x,y) - c(y,x)} = (l/2){c(x, y) - c*(x,j/)}, the previous 
expression is equal to 

i 5>/(z)(/-P)/ 2 (x) - i £m(x)(/-P*)/ 2 (x) = 0, 

xEE xeE 

where P represents the operator in L 2 (M) defined by (Pg)(x) — ^2 V £eP( x i J/)ff(j/)> 
and where P* stands for the adjoint of P in L 2 (M). This proves (14.21) . 

This orthogonality permits to restate Lemma 14.21 in a slightly different form, 
quite useful in some cases. 

Lemma 4.4. Fix a pair of points a ^ b in E. Then, 

cap({a},{6}) = inf inf {£>(/) + ||T, - <^|| 2 } , 

where the infimum is carried over all junctions f : E —> R such that f(a) = I, 
/(6) = 0. 

We are now ready to prove Theorem 12.71 

Proof of Theorem \2. 71 We proceed in two steps, collapsing each set at a time. Fix 
two disjoint subsets A, B of E and recall the notation introduced around (|3.9p . We 
first prove that 

inf inf \\® F - Lp\\ 2 = inf inf - VIIa , ( 4 - 7 ) 
F v /V 

where the infimum on the left hand side is carried over all functions F : E — > R 

constant over A and flows ip such that (div <p)(x) — 0, x £ A c , YlxeA^v f)( x ) = 0j 

while on the right hand side || • m represents the norm associated to the scalar 

product introduced in (|2.I2j) on the set E a for the process X t and the infimum is 

carried over all functions / : Ea — > R and divergence free flows i/j on Ea ■ 

Consider a function F : E — > R constant in A and a flow ip on E such that 

(div </?)(£) = 0, x <E A c , XLeA^iv vX^) = 0- Recall the definition of the function 

/ : Ea —> R introduced below (|3.8|) and let ^ be the flow on Ea given by 

V'fof) = <P(?,y) , VX^f) = E^' 2 ^' ^M/ ^ A c . 

yeA 
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One checks that ^ is a divergence free flow. Moreover, by Schwarz inequality, 

Pz-V'lli < ll^-^f • 

It follows from this estimate that the left hand side of (|4.7|) is greater than or equal 
to the right hand side. 

Conversely, fix a function / : Ea — > R and a divergence free flow ip on Ea- Let 
F : E — > R be the function defined above (|3.8|l , and let ip be the flow in E given by 

p(af, y) = ip(x,y) , x,y€A c , <p(z,w) = 2f(p)c a {z,w), z,weA, 

Z^26A ^V^) Z i L zSj4 J 

One checks that (div <p)(x) = 0, x G A c , that X^eA^iv ^X 2 -) = 0> and that 
[|$f — i/3|| = ||$/ — a- Therefore, the left hand side of (|4.7[) is less than or equal 
to the right hand side, proving claim (|4.7[) . 

We are now in a position to prove the theorem. Fix a site x £ E and a set ^4 ^ x. 
By (|3.10|) . cap({a;}, A) — cap({a;}, {D}). The assertion of the theorem when the set 
B is a singleton follows from Lemma 14.21 and (|4.7j) . The general case is proved 
analogously by first collapsing the set A and then collapsing the set B. □ 

We conclude this section with a bound on the capacity in the denumerable case. 
Assume that E is a countable set, fix a site x € E and a set B $ x, with B c finite. 
Then, 

cbq{{x},B) < (4.8) 
1 ^ c a (x,y) 2 „ l2 , > c Q (x,z) 2 -( 



inf 



x,y£B" x£B c ,z£B 



where the infimum is carried over all functions F : E — > R such that .F(x) = 1, 
F(z) = 0,z6B. 

Indeed, recall the notation introduced around (|3.13p . Denote by E the set B c U 
{D}, where D is an extra site added to E. Let {Xt : t > 0} be the process obtained 
from by collapsing the set B to the point D, and let D, £, c(x, y) and cap be the 
associated Dirichlet form, oriented bonds, conductances and capacities, respectively. 

By (f3~T4|) . cap({a;},B) = cap({a;}, {0}). Fix a function F : E -> R which 
vanishes on i? and is equal to 1 at x, and let / : E — > R be given by f(y) = F(y), 
y G B c , /(0) = 0. Since c a (x, y) is a divergence free flow, by Lemma T4.41 

cap(0,D) < D(f) + \ V _^^{c a ( a; , y )[/(x)+/( 2/ )]-2c a ( a ;,2/)} 2 . 

Clearly, £>(/) = D{F). On the other hand, by (|3.13p if the arc (x, y) is contained in 
B c , we may replace c s (x, y), c a {x, y) and / by c s (x, y), c a (x, y) and F, respectively. 
In contrast, for an arc (x,d), x G B c , since Ct(a;,D) = XzeB c *( 2 '' z )' ^ = a,s, and 
since /(D) = = F(z), z G _B, by Schwarz inequality, 

{c Q (x,D) [/(*) + /(D) - 2] } 2 = {^ Ca (a;,z)[F(x)+f(z)-2]} 2 

< E Ss r l£ T^(*) + n*) - 2] 2 E c *(z> z ) ■ 

zeB Cs ^ z ) zeB 
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Since < F < 1, F(x) + F(z) — 2 is absolutely bounded by 2. Putting together all 
previous estimates we derive (|4.8|) . 



5. Recurrence criteria 

It is well known that in the reversible case the Dirichlet and the Thomson prin- 
ciple provide powerful tools to prove the recurrence or the transience of irreducible 
Markov processes evolving in countable state spaces. In this section, we examine 
this matter in the non reversible case. 

Consider a irreducible Markov process {X t : t > 0} on a countable state space E 
satisfying the assumptions of the beginning of Section [2l We assume, in particular, 
the existence of a stationary state /i. 

It is well known that the Markov process X t is recurrent if and only if there exist 
a site G E and a sequence of finite subsets B n containing and increasing to E, 
B n C B n+ i, U n B n = E, such that 



By (|2.3[) , for any finite set B containing the site 

1 



lim P [T B c < T+] = . 
ntaining the site 0, 
[Tb c < T+] = cap(0, B c ) 



M(0) 

Hence, the Markov process X t is recurrent if and only if there exist a site G E 
and a sequence of finite subsets B n containing and increasing to E such that 

lim cap(0,S^) = . (5.1) 

The proof of the recurrence is thus reduced to the estimation of the capacity between 
a site and the complement of a finite set. This problem can be further simplified 
by collapsing the set B^ to a point, as we did in Section [3] 

The first two results follow from the previous observation and the bounds stated 
in Lemmas 12. 51 and 12.61 Recall that {X£ 1 1 > 0} stands for the reversible version of 
the process X t whose generator is given by S. 

Lemma 5.1. Let {X t \ t > 0} be a irreducible Markov process on a countable state 
space E which admits a stationary measure. The process is transient if so is the 
Markov process {X£ 1 1 > 0}. 

Lemma 5.2. Let {X t \ t > 0} be a irreducible Markov process on a countable state 
space E which admits a stationary measure. The process is recurrent if its generator 
satisfies a sector condition and if the Markov process {X t s 1 1 > 0} is recurrent. 

Cycle random walks with bounded cycles, [13) . |12j . mean zero asymmetric ex- 
clusion process [16] , or asymmetric zero range process on a finite cylinder [9] are 
examples of non reversible Markov processes which satisfy the sector condition. 

Lemma 5.3. Let {X t \ t > 0} be a irreducible Markov process on a countable state 
space E which admits a stationary measure. The process is recurrent if the Markov 
process {X£ \ t > 0} is recurrent and if 



E 



c a (x,y) 2 

< oo 



c s (x,y) 
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Proof. Fix e > and a site G E. By assumption, there exists a finite set A 3 
such that 

c a (x,y) 2 < ^ 

(x,y)££ 
{x,y}£A 

By jUJ), for all subsets B of £ such that A C B c , B c finite, cap s (A,B) < 
^^g^ cap s ({cc}, B). Hence, since the process X t s is recurrent, by (|5.1[) and by 
Lemma [221 there exists a finite set B c D A such that cap s (A, B) < e. 

Denote by B ■ — > K the equilibrium potential associated to the reversible 
process X s : VX B \x) = P S X [T A < T B ]. Since D(V£ iB ) = cap s (A,B), by construction 
of B, D(Vi B ) < e. Therefore, by g^J with F = V% B , cap(0, B) is bounded above 
by 

a(x,y) 2 2 , c a (x, z) 2 



x,y£B" xeB c ,z£B 

Since F| B is identically equal to 1 on A, the previous expression is less than or 
equal to 



E 



c a {x,y) 2 + 2 c a{x,y) 2 



c s (x,y) c s (x,z) 

xGA,y£A c x,yeA° 

By definition of the set A, this expression is bounded by 5e, which concludes the 
proof of the lemma. □ 

5.L Random walks with self-similar rescaled invariant potential. In [8], 
Durrett built from a random potential, with a large scale self-similarity property, 
a reversible nearest-neighbor random walk on 1 d for which Sinai random walk is a 
special case when d = 1. He proved that such a random walk is recurrent under 
simple and natural assumptions on the scaling limit of the potential. Note, however, 
that such a random walk could never be a particular case of classical randoms walks 
in random environment in dimension d > 2 due to the reversibility condition. 

We want to point out here that the key feature of this model is not the reversibil- 
ity but, as Durrett suggested, the existence of a strongly fluctuating invariant mea- 
sure. The reason for the restriction to the reversible was that it allowed the use 
of the Dirichlet principle. Our extended Dirichlet principle permits to reproduce 
Durrett's argument with only assumptions on the invariant measure and without 
the reversibility hypothesis. 

Consider a discrete time, nearest-neighbor random walk {X n \ n > 1} on Z d with 
random transition probabilities p(x,y) : Q — > [0,1], and assume the existence of a 
(random) invariant measure [i. We define the invariant potential V : Z d — > K by 

fj,( x ) = e - v{x) x e Z d , 

and assume, without loss of generality, that V(0) = 0. The relation between the 
random potential and the invariant measure in [5] is not exactly the same, but this 
definition makes our point more clear. We can extend V into a continuous function 
V : M. d — > K and see it as a random variable in C(K d ,K), the space of continuous 
functions from R d to R equipped with the topology of uniform convergence on 
compact sets. We assume that there exists a > and a random variable W : 51 — >• 
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C(E d , E) such that A a V(\ • ) converges in law to W( ■ ) when Afco. Hence, W 
is a self-similar d-dimensional process and we have the following result. 

Lemma 5.4. If there is almost surely a > such that the connected component of 
the origin in {x G R d : W(x) < a} is bounded, then X is almost surely recurrent. 

We refer to [5] for examples of (reversible) processes which satisfy such hypothe- 
ses. Even though we could relax Durrett's reversibility hypothesis, it is not clear 
how to build non artificial irreversible examples in which one has enough control on 
an invariant measure to check the assumptions on V. One can start, for example, 
as in |8l, with a random potential V with stationary increments, and build the 
reversible random walk inside this potential to have u. as invariant measure. We 
may then add some irreversibility by superposing to this reversible dynamics some 
drift along cycles on the level sets of V, keeping \x as an invariant measure. 

Proof of Lemma \5.4{ We follow closely Durrett's proof. First, following Skorohod 
[14) . we can build on the same probability space random variables Vi, V2, ■ ■ ■ with 
the same law as V and such that n~ a V n (n ■ ) converges almost surely in C(K d ,R) 
to W( ■ ). Define C a , a > 0, as the bounded connected component of the origin in 
{x e R d : W(x) < a}, and set 

G n = (nC a ) n Z d , cLG„ - {x e G n : By # G n , \\x - y\\ = 1} , 

where || • || stands for the Euclidean norm. 

We claim that /x„(9_G n ) converges almost surely to 0, where fJ- n (x) = e~ Vn ( x \ 
xgZf Indeed, by assumption there are r, R > such that 

C a C [-R, R] d , W(y)>a/2 

for all y G B(z,r), z £ dC a , where B(x,r) stands for the ball centered at x with 
radius r and dC a for the boundary of C a . Therefore, almost surely, for n large 
enough, 

(i n (d-G n ) = erVn[x) ^ \Gn\e- {a/i)na < n d {2R) d e- {a '^ n \ 

xed-G n 

which proves the claim. 

For any finite subset B of Z d which contains the origin, since /z(0) = e~ v ^ = 1, 
we have 

Fo[r + = 00] < M0)P [t + > r+ c ] = cap(0,B c ) . 

By taking the test function 1{B} in (I4.8P we obtain that cap(0, B c ) is bounded 
above by 

E +4 E C ir£rT- 5 E c s (x,y) < 5K8-B) . 

Since /i(0) = 1 we also have cap(0,B c ) < 1, thus cap(0,B c ) < 5fi(d-B) A 1. Now, 
for any k > 0, 

P {t+ = +00) < min {cap(0, B c ) > : OeB C [-k, k] d ) 
< min 5u(d-B) A 1. 

QeBG[-k,k] d 

Since V n has the same distribution as V, taking expected values with respect to 
the environment, denoted by E, by the monotone convergence theorem, we obtain 
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that, for all n > 1, 



E[P k n + = +ooll < lim E min 5u(d_B) A 1 
J fc^oo LoeBc[-fc,fc] d 



lim E 



min 5fi n (d-B) A 1 
oeBc[-fc,fc] d 



E 



lim min bfi n {d-B) A 1 
fc->oo oeBc[-fe,fe]< i 



< E 



5n n (d-G n ) A 1 



Thus, by the dominated convergence theorem, 
E[F [r + = +oc 



< lim E[5u„(5_G„) A ll 



E 



lim 5{i n (d-G n ) A 1 

n— f+oo 



, 



which proves that the process is almost surely recurrent. 



□ 



5.2. Two dimensional random walk in asymmetric random conductances. 

The most natural way to generalize the classical random conductance model on 
a graph may be the following. To define the asymmetric conductances c(x, y) on 
each arc (x, y) we superpose symmetric functions c s (x, y) and a divergence free flow 
c a (x,y) with the restriction that \c a \ < c s to end with nonnegative conductances 
c(x,y). 

More precisely, consider a family T of finite cycles 7 on a countable graph (E, £), 
and a family of nonnegative random variables Z 7 , 7 G I\ such that for each (x, y) G 

X! Z 7 1X7(^)1 < 00 > 

where x-y is the divergence free flow introduced in (12.14[) . Define the divergence 
free flow c a by 

c a (x,y) = ^2z 7 xi(x,y) > (x,y)e£. 

7er 

There are two natural ways to define symmetric conductances in this context. 
Consider a family of nonnegative random variables {Yr Xty \ : {x 7 y) G £} such 
that = Y {y . x) . We may set c s (x,y) = Y {x ^ y) + \c a (x,y)\, or c s {x,y) = 

Y{x, y ) +E 7G r Z 7 \x 7 {x,y)\- 

In the special case of the two dimensional lattice, we can decompose each flow 
associated to a finite cycle as a linear combination of elementary flows associated to 
cycles of length 4. For ir. £ Z 2 , denote by 7^ the cycle (x, x + ei, x + ei + e2, x + e2, x), 
where e%, e2 stands for the canonical basis of R 2 . A flow x-y associated to a finite 
cycle 7 can be written as 

X7 = ^ ] ^7,7x ^7x 1 



1 (resp. —1) if the cycle "f x is contained in the interior of 7 and the 

if the cycle j x is 



where W 7i , 

cycle 7 runs counter-clockwise (resp. clockwise), and W 7)7!E 
not contained in the interior of 7. 

Denote by E expectation with respect to the random variables Z 1 and assume 
that 

\ ^ T7i r r7 1 \ jrr I , _ _ r-__ -11 „ njd 



7er 



< 00 for all x G Z 
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In this case W lx := X) 7 er %i W ltlx is almost surely well denned for all x £ Z d and 
so is the divergence free flow c a given by 

Ca = J2 W ^ *T- ■ ( 5 - 2 ) 

Note that each arc (x, y) belongs to exactly two elementary cycles, denoted by 
7 ± (a;, y) and characterized by the fact that X~/±(x,y)( x i V) = il- With this notation, 
for any arc (x,y), c a (x,y) = W 1+{x ^ y) - W^-^ y y 

Lemma 5.5. Suppose that 



sup E 

(x,y) 



c s (x,y) + 



< oo , 



Cs(x,y) 

where the supremum is carried over all arcs. Then, the random walk is almost 
surely recurrent. 

Proof. Let = [— n,n} 2 , n > 1, consider a function f n : 1? — > R such that 
/n(0) = 1, f n {x) — for x £ _B„, and a divergence free flow ip n = J2 X a xX-y !C i where 
the sum is performed over all i£Z 2 for which the elementary cycle j x is contained 
in B^- Repeating the proof of (|4.8|) and keeping in mind that c a is absolutely 
bounded by c s , we obtain that 

cap(0,S„) < £>(/„) + \ V / |c a (x, y) [f n (x) + f n (y)] - (a, y) | ■ 

2 ^— ( c s (x, y) L J 

Consider the divergence free flow ip n given by 

<Pn = 7j^2 F n{lx)W l!l; Xix , where F n (^ x ) = £ f n (z) , 

x zG7x 

and where the sum is carried over all sites x in Z 2 for which the elementary cycle 
j x is contained in _B^. By the previous bound, 

cap(0,S„) < £>(/„) + - — ( Ac a (x,y)[fn(x) + f n (y)} 

As we know, D(f n ) = (1/2) X^ez 2 c s{ x > y)[fn (v) ~ fn{x)} 2 . On the other hand, 
it follows from the definitions of the asymmetric conductance and the divergence 
free flow ip n that c a (x,y)[f n (x) + /„(y)] - fn(x,y) is equal to W 7 +( Xy y){f n (x) + 
f n {y) - (l/2)F n ( 7 +(z,y))} - W^ {x>y) {Ux) + f n (y) - (l/2)F n ( 7 -(x,y))} if the 
arc (x,y) does not belong to one side of the square B^. The absolute value of 
this difference is bounded above by |W 7 +( X ty \\ max e67 +( Ij9 ) |/n(e + ) — /ri( e ~)l + 
max e£-y-(x ,y) /n(e + ) — f n (e~)\. If the arc (x,y) belongs to one side of 
the square B^ , taking advantage of the fact that /„ vanishes outside B^ , we obtain 
a similar formula with an extra factor 2. In conclusion, cap(0, B n ) is bounded above 

by 

4 £ {c a (x, y) + fa^± [IfiW } max[/K ( e+ ) - /m(e -)]2 , 

where the maximum is carried over all arcs e in 7~(x, y) U 7 + (x, y). 
Let 

/.W-('- ii ^f 2 )M[-.-l'}W. 
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where ||x||ao = max{|xi|, \%2\}, x — (xi,X2). It follows from this choice and from 
the assumption of the lemma that 

lim E[cap(0,5„)] = . 

n— >oo 

In particular, there exists almost surely a subsequence (B nk : k > 1) such that 
linifc-^oo cap(0, B nk ) = and the almost sure recurrence follows. □ 

We conclude with an example which satisfies the assumptions of the previous 
lemma. Suppose that the random variables Z 1 are independent Poisson variables 
with parameter A' 7 ', < A < 1/3, and that the random variables Y( x ,y) have a 
common distribution bounded away from and with a finite first moment. Let c a 
be given by (|5.2[) and let c s (x, y) = Y( x ,y) + \c a (x, y)\- We claim that the hypotheses 
of the previous result are fulfilled. 

Indeed, by assumption there exists S > such that c s (x,y) > Yt x ,y) > S > 
almost surely. Therefore, to show that the assumptions of the previous lemma are 
in force we need only to prove that 

sup E[c a (x, y)] < oo and sup E[W^J < oo . (5.3) 

0,1/) x£Z d 

Since \c a (x,y)\ < W^+^y) + Wj-( x>y ), for every arc (x,y), 

E[c s (x,y)] < E[Y(x,yj\ + £ ^^.jE^] . 

P=± 7 er 

By assumption, the first term on the right hand side is bounded uniformly over 
(x,y), while the second term is less than or equal to 53fc>4 A because there 
are at most 4 • 3 k ~ 1 self-avoiding walks of length k and because a cycle of length 
k containing in its interior an elementary cycle must cross a line parallel to one of 
the axis in at most 2k points. This proves the first bound in (|5.3[) . To prove the 
second bound, fix an elementary cycle j x . By definition of the random variables 

E[WI] = E[iy 7 J 2 + Y, Xhl Ki* ■ 

The first expectation has been estimated above, while the second one can be esti- 
mated in the same way. This concludes the proof of (|5.3[) . 
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